Replace legacy pipeline with token-efficient semantic index - #1
Merged
Conversation
fajarnugraha37
marked this pull request as ready for review
July 16, 2026 22:13
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the provider-heavy v1 workflow with DocGen 2.0, a breaking semantic-index architecture for repositories of any language, framework, library, runtime, infrastructure stack, or architecture.
Providers receive bounded, content-addressed context packs. Deterministic code owns source discovery, indexing, checkpoints, validation, quality gates, budgets, telemetry, and publishing.
Technology-neutral architecture
DocGen does not assume JAX-RS, Jersey, MyBatis, SQL, Kafka, Java, or any other particular stack. It supports applications, libraries, CLIs, jobs, plugins, infrastructure, data pipelines, embedded systems, monoliths, services, and mixed repositories.
The semantic index always records generic file artifacts and source chunks, then adds cross-language hints when available:
Unknown technologies remain supported through bounded source chunks rather than being rejected.
Phase 2A — runtime stabilization
Phase 2B — correctness validation
A new deterministic quality engine validates:
Publishing now fails when source, model, page, traceability, link, evidence, or audit inputs are stale.
Token-efficiency architecture
.gitignore/.docgenignore/ binary-aware inventoryValidation
The regression suite now contains 20 tests covering:
The complete suite and installer dry run pass on:
Breaking migration
Version remains
2.0.0; Node.js 22.5+ is required fornode:sqlite.Existing repositories run:
Migration preserves generated docs,
.docgenignore, project identity, selected provider settings, and ignore/binary limits. Legacy workflow state is archived rather than interpreted as a current checkpoint.Status
Phase 2A and Phase 2B implementation is complete. The PR remains draft and unmerged pending validation against the real target repository and the Phase 2C cold/warm/incremental token benchmark.